Adventures in Ansible and Vagrant

NB: This is a work in progress, probably doesn't represent best practice, but this topic is thorny and poorly-documented, so I thought it best to actually talk through the discovery process.

I use Ansible to manage my personal infrastructure. It means I can keep config in version control, and migrate between VMs / servers easily.

Like most home users, I don't always follow best practices. If my self-hosted Nextcloud doesn't reach five-nines uptime it doesn't actually matter that much. So I don't have a proper test / staging environments. I have assumed that the effort required to do so outweighs the benefit.

Recently, after a Traefik bug caused lots of people's production environments to break, I thought I'd revisit this assumption. After all, tooling for throwing up ephemeral VMs for a test environment have been around for a while.

I want to spin up test VMs on my laptop using libvirt, rather than using a third-party cloud provider. For this reason, I decided to look at Vagrant - Debian provides Vagrant packages which default to libvirt as a backend. It also seems pretty lightweight compared to Terraform, and particularly compared to OpenStack.

My first assumption is that I'd want to use ansible to call Vagrant to set up the test VMs. Asserting the existence of the VMs seemed like a sensible step. However, after discussion with fellow geeks on IRC, I accepted that orchestration and configuration management are probably better considered separate concerns for the time being.

Splitting Environments

My ansible repository has a fairly standard layout:

ansible.cfg
inventory
group_vars/
host_vars/
personal.yml
roles/
vault_key ->

Setting up a separate production environment initially is as simple as moving host_vars and inventory into a subdirectory called production, which will live alongside a subdirectory called test.

ansible-playbook -i production personal.yml

works pretty much exactly as it did before. Now I need to populate the directory test.

Integrating the Tools

One major concern I have is duplicating configuration between Ansible and Vagrant. This will inevitably lead to mismatches which will be a pain to debug.

An obvious example is IP addresses for VMs - Ansible won't know how to talk to a Vagrant-spawned VM by default. I can let Vagrant edit /etc/hosts, but that feels inelegant to me. I can also hard-code IPs but that means maintaining the mapping in both systems and can lead to things getting out of sync. I'd rather have a way for Ansible to discover Vagrant-created VMs automatically.

Both Ansible and Vagrant have guides on integrating with the other. These both talk about Vagrant's Ansible "provisioner" being used to run a playbook against a new VM when it is set up, or when you run vagrant provision.

This feels a bit wrong to me - I want to use the same ansible-playbook command against both my test (Vagrant-provisioned VMs) and production environments. So I'm interested in the Ansible-compatible inventory file created by Vagrant's ansible provisioner. This is created relative to the Vagrantfile, and I want it to be picked up by ansible-playbook. It seems that the easiest way to do this is to actually keep my Vagrantfile for my test environment inside the test/ subdirectory in my Ansible repo.

Creating the VM with Vagrant

I hack a default test/Vagrantfile as follows:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  # The most common configuration options are documented and commented below.
  # For a complete reference, please see the online documentation at
  # https://docs.vagrantup.com.

  config.vm.box = "debian/bullseye64"
  config.vm.hostname = "router"
  config.vm.define "router"

  config.vm.synced_folder ".", "/vagrant", disabled: true

  # Run a basic playbook once the VM is set up
  # Mostly to generate an inventory
  config.vm.provision :ansible do |ansible|
    ansible.verbose  = "v"
    ansible.playbook = "../vagrant.yml"
  end

end

I disable NFS sharing to the VM to avoid having to type in a root password on vagrant up, and because I don't need it - everything I want to do to the VM should be on the host and applied via ansible.

The vagrant.yml playbook referenced is just a stub - I only want to use Vagrant's ansible provisioner to generate an inventory entry. Now when I create the VM with vagrant up I get the usual Vagrant bumph followed by:

==> router: Running provisioner: ansible...
    router: Running ansible-playbook...
PYTHONUNBUFFERED=1 ANSIBLE_FORCE_COLOR=true ANSIBLE_HOST_KEY_CHECKING=false ANSIBLE_SSH_ARGS='-o UserKnownHostsFile=/dev/null -o IdentitiesOnly=yes -o ControlMaster=auto -o ControlPersist=60s' ansible-playbook --connection=ssh --timeout=30 --limit="router" --inventory-file=$HOME/repos/personal-ansible/test/.vagrant/provisioners/ansible/inventory -v ../vagrant.yml
No config file found; using defaults

PLAY [all] *********************************************************************

TASK [Gathering Facts] *********************************************************
ok: [router]

TASK [Say hello] ***************************************************************
ok: [router] => {
    "msg": "Hello world!"
}
PLAY RECAP *********************************************************************
router                     : ok=2    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

This tells me that it's created the inventory file, under my test/ environment directory.

Using the Vagrant inventory in Ansible

Now to see if Ansible can pick this up:

$ ansible-playbook -i test personal.yml  --list-hosts
[WARNING]:  * Failed to parse $HOME/repos/personal-ansible/test/Vagrantfile with yaml plugin: We were unable to read either as JSON nor
YAML, these are the errors we got from each: JSON: Expecting value: line 1 column 1 (char 0)  Syntax Error while loading YAML.   did not find
expected <document start>  The error appears to be in '$HOME/repos/personal-ansible/test/Vagrantfile': line 9, column 3, but may be
elsewhere in the file depending on the exact syntax problem.  The offending line appears to be:     config.vm.box = "debian/bullseye64"   ^ here
[WARNING]:  * Failed to parse $HOME/repos/personal-ansible/test/Vagrantfile with ini plugin: $HOME/repos/personal-
ansible/test/Vagrantfile:4: Expected key=value host variable assignment, got: do
[WARNING]: Unable to parse $HOME/repos/personal-ansible/test/Vagrantfile as an inventory source
[WARNING]: Unable to parse $HOME/repos/personal-ansible/test as an inventory source
[WARNING]: No inventory was parsed, only implicit localhost is available
[WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match 'all'
[WARNING]: Could not match supplied host pattern, ignoring: docker_hosts
[WARNING]: Could not match supplied host pattern, ignoring: routers

playbook: personal.yml

  play #1 (docker_hosts): docker container hosts        TAGS: [docker,docker_host]
    pattern: ['docker_hosts']
    hosts (0):

  play #2 (docker_hosts): docker containers     TAGS: [docker,docker-containers]
    pattern: ['docker_hosts']
    hosts (0):

  play #3 (routers): router     TAGS: [router]
    pattern: ['routers']
    hosts (0):

  play #4 (all): users  TAGS: [users]
    pattern: ['all']
    hosts (0):

There are two things going on in these warnings. Firstly, it's trying to parse my Vagrantfile as an inventory source, and secondly, it's not finding the Ansible inventory created by Vagrant in test/.vagrant/

The first is annoying but trivial to fix; I simply add the following to my ansible.cfg:

inventory_ignore_extensions = Vagrantfile

The second, I assume, is caused by Ansible not looking inside a directory that starts with a period. If I copy the Vagrant-generated inventory file into test/ itself, then --list-hosts works as expected. However, this obviously won't stay in sync with future Vagrant-generated changes to that inventory. Fortunately we can fix it with a symlink created inside test/:

ln -s .vagrant/provisioners/ansible/inventory/vagrant_ansible_inventory 01_vagrant_ansible_inventory

Assigning the Vagrant VM to a role

I also create a separate file 02_ansible_groups in the test/ folder that puts the Vagrant VMs into Ansible inventory groups, as this is how I keep track of which roles are applied to which systems.

[routers]
router

Tidying up

At this point, I try to commit the work I've done so far to a git branch, but I notice that it wants to include the test/.vagrant/ folder, which is ephemeral data which will vary across systems. So I add .vagrant to my .gitignore file to avoid that.

Summary

At this point, I have a Vagrantfile which creates a VM called router, and an Ansible inventory file for it, which Ansible can read. This is good enough for me to develop my role against. Right now my filesystem tree looks like this:

ansible.cfg
group_vars/
personal.yml
production/
\- host_vars/
\- inventory
roles/
test/
\- 01_vagrant_ansible_inventory ->
\- 02_ansible_groups
\- Vagrantfile
\- vagrant.yml
vault_key ->

I will need to do more Vagrant and Ansible work - the next stage is to get my Vagrantfile to set up a second VM (no point having a router if there's nothing for it to route to!) with some complicated networking. But that's beyond the scope of this article, which is just about getting an Ansible / Vagrant workflow I'm happy with.

links

social